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A  Reversal  Argument  for  Storage 
Models  Defined  on  Semi-Markov  Processes 

by 

Erie  S.  Tollar 

Abstract 

For  many  storage  models  defined  on  some  semi-Markov  process  X(t),  the 
asymptotic  distribution  of  the  imbedded  discrete  time  process  can  often  be 
determined  by  exploiting  the  properties  of  the  dual  of  the  underlying 
Markov  chain  X^,  which  effectively  reverses  the  process.  If  this  is  the 
case,  a  technique  is  given  which  under  certain  regularity  conditions  shows 
the  asymptotic  distribution  of  the  entire  .  cntir.uous  time  process  can  be 
obtained,  and  is  equal  to  an  altered  version  of  the  "reversed"  discrete 
time  process.  It  is  shown  this  method  not  only  can  be  applied  to  models 
where  the  asymptotic  distribution  was  previously  unknown,  but  can  also 
improve  upon  characterizing  many  of  the  results  for  models  in  which  the 
asymptotic  behavior  is  obtained  by  a  renewal  argument. 


1.  INTRODUCTION 


In  storage  models,  the  concept  of  using  an  underlying  Markov  renewal 
process  to  allow  for  some  dependency  of  structure,  as  well  as  continuity 
of  time,  has  seen  widespread  usage.  In  such  models,  the  investigation  of 
the  limit  behavior  of  the  contents  in  storage  as  time  tends  to  infinity 
has  always  been  one  of  the  more  important  aspects  of  the  model,  and  there 
have  been  a  wide  variety  of  techniques  used  in  the  literature  to  determine 
the  limit  behavior. 

When  the  structure  is  such  that  the  amount  in  storage,  when  coupled 
with  the  state  of  the  underlying  semi-Markov  process,  is  itself  a  semi- 
Markov  process  on  some  arbitrary  state  space,  the  general  theory  of  semi- 
Markov  processes  on  arbitrary  state  spaces  (for  example,  see  Cinlar  (1969), 
Athreya,  McDonald  and  Ney  (1978a,  1978b),  Athreya  and  Ney  (1978),  Kesten 
(1974),  and  Nummelin  (1978))  can  be  exploited.  This  technique  was  success¬ 
fully  exploited  by  Puri  and  Tollar  (1985)  to  determine  the  limit  behavior 
of  a  popular  storage  model. 

Another  popular  technique  is  to  "reverse"  the  process  by  looking  at 
the  dual  Markov  renewal  process  (for  a  summary  of  the  full  power  of  the 
dual  process,  see  Keneny,  Snell  and  Knapp  (1976)).  While  this  method  has 
proven  itself  useful  in  the  limit  behavior  in  those  cases  where  the  con¬ 
tents  in  storage  require  some  normalization  (see,  for  example,  Puri  and 
Woolford  (1981)),  it  has  failed  in  those  cases  where  no  normalization 
is  required. 


In  this  paper,  under  certain  assumptions  on  the  definition  of  the 

i 

l 

storage  model,  we  present  a  technique  which  allows  one  to  "reverse" 
the  continuous  time  process  to  obtain  results  in  those  cases  where  no 
normalization  is  required.  Because  of  the  assumptions  imposed  on  the 
model,  these  results  can  also  be  considered  as  an  extension  of  the 
theory  of  semi-Markov  processes  on  arbitrary  state  spaces  (although 
there  is  admittedly  more  structure  on  our  state  space  than  the  pre¬ 
viously  cited  authors  prefer  to  allow). 

Let  J  be  a  subset  of  the  integers,  and  (X^,  n =  0,  1,  2,  . . . }  be 
a  stationary,  irreducible,  aperiodic,  positive  recurrent  Markov  chain 
with  transition  matrix  P=  (P  — )  for  i,  j  e  J,  and  with  stationary 
measure  We  then  define  times  0=1-.  £T.  <TV..  such  that 

U  1  L. 


{ (Xn»  Tn)  ,  n  =  0,  1,  2,  . .  .  }  is  a  Markov  renewal  process  with  semi 
Markov  matrix  A(t)  =  (.‘..^(t)),  ■.:bere  f or  i ,  j  eJ,  t>0, 


P<VJ-  VT„-1«' V  V  V  V 


■  Xn-  l  =  i>=AiiW 


(ree  £inlar  (1975)  for  details).  For  all  ieJ,  define  the  expected 
sojourn  time  in  state  i  by 


and  define  the  average  sojourn  time  of  the  process  by 


For  all  t>  0,  define  the  number  of  jumps  by  time  t  by 

,M(t)  =  sup  {n:  st). 

We  define  another  Markov  renewal  process  {(Xn,  T  )},  independent 
of  {(Xn>  Tn)},  with  semi-Markov  matrix  A(t)  =  (A^ (t))  defined  for  each 

i,  j  e  J  by 

A.  .(t)  =  ir.rr^A.  .  (t) , 
lj  31  3 1 1  J 

and  let  have  initial  distribution  3. 

Definition .  {(X  ,  T  ) }  as  defined  above  is  called  the  dual  Markov 

renewal  process  of  {(X  ,  T  )},  and  (X  }  is  called  the  dual  Markov  chain. 

n  n  n  - 

Finally,  for  each  iej,  wt  associate  a  sequence  of  i.i.d.  random 

variables  {"  ( i )  ,  r.  =  0 ,  1.  2,  ...}  on  'orn  ar’  itrarv  state  soacc.  where 

{U  (i)}  is  independent  of  {(X  ,  T  )>  and  {(X  ,  T  )},  and  of  all 
n  nn  vnn 

for  j  *i.  We  then  define  the  contents  in  storage  at  jump  n 
recursively  for  some  function  f  by 

Zn(x)  =  f(Znl(x),  Un(Xn)),  (1.2) 

where  ZQ(x)  = x. 

For  simplicity,  we  will  assume  the  function  f(*)  is  real-valued. 

To  define  the  amount  in  storage  at  time  t,  we  let 

2(t)  =  ZN(,)(zn>- 

If  we  define  a  sequence  of  functions  recursively  by 


we  can  represent  Zn(x)  by 


_  *00 


Vx)*f  (xi  W’  U2(X2),  ....  Un(Xn)). 


Define  Z  (x)  by 


_  *(n) 


Zn(x)-f'“'(x;  _  jfV.  -  P  •  WVP-  W’- 

For  convenience,  let 


g(fl)(x;  ilfi2,  ....  in)  =  f(n)(x;  (Un_2(i2)»  •••>U0(in)).  (1.3) 


Note  frc::  Keceny  ..Snell  and  Knapp  (1176)  that  if  XQ  has  initial  dis- 


tribution  *  ,  Z  (x)=Z  (x) ,  for  all  a. 
n  n 


Wc  will  assume  throughout  the  paper  the  following  condition  on 
the  storage  model  of  (1.2):  for  any  arbitrary  distribution  of  Xn>  for 
each  B > 0,  and  each  c  >  0,  there  is  an  N  where  for  all  n>  N, 


P(  sup  |  Z  (0)  -  Z  (x)  l>  fi)  <  c. 
I  x  l<  *  n  n 


(1.4) 


As  such,  the  initial  amount  of  the  contents  in  storr»»-''  is  "niformly 
forgotten  as  n  tends  to  infinity. 

The  importance  of  the  condition  anu  the  usefulness  of  the  revers¬ 
ibility  in  discrete  time  n  can  be  seen  from  the  following  theorem. 

This  theorem  is  stated  without  proof,  for  even  though  it  is  not  explic¬ 
itly  stated  in  the  literature,  the  techniques  required  to  prove  it  are 
well  established  (see  for  example  Puri  and  Wool  ford  (1981)). 


THEORE'1  1.1.  Tf  condition  (1.4)  ij^  satisfied,  then  the 

lim  P(X  =  i,  Z  (x)  £  y)  exists  for  all  x.  and  for  all  initial  distributions 

of  X„,  if  an  only  if  limP(X  =i,  Z  (0)  £  y)  exists,  in  which  case  for  all 

continuity  points  y  of  lim  P(X  =i.  Z  (n)  <  v)  ,  for  all  x.  and  all  ieJ 

lim  P (X  =  i  ,  Z  (x)  <  y)  =  lim  P  (X  =  i  ,  Z  (0)  <v). 
n-*»  n  n  n-x“  ' 

Therefore,  in  discrete  time,  one  can  either  examine  the  original  process 
or  the  dual  process,  whichever  is  more  convenient. 

It  is  easy  to  see  that  {X  ,  Z  (Zn) ,  T  }  is  itself  a  Harkov  renewal 

n  n  u  n 

process.  However,  the  state  space  cf  (X^,  Zn(Z  )}  need  not  he  denumerable 

Therefore,  even  if  it  is  possible  to  establish  that  (X  .  Z  }  converges  in 

n  n 

distribution  as  n  tends  to  infinity,  it  need  not  fellow  that  (X(t),  Z(t)} 
converges  in  distribution.  The  substantial  body  work  on  semi -Markov 
processes  on  arbitrary  state  snaces  cited  previously  is  of  little  help 
in  proving  what  annears  should  he  true:  as  lone  as  R  < *  in  (1.1),  then 
(X^,  Z^)  converging  should  imply  (X(t),  Z(t)>  converges . 

It  will  be  shown  in  section  2  that  under  certain  conditions  a  re¬ 
versibility  argument  can  be  applied  to  determine  the  convergence  in 
distribution  as  time  tends  to  infinity.  Section  7>  is  then  devoted  to 
applications  of  the  results  in  section  2  to  some  examples  or  storage 


models . 


2.  THE  REVERSIBILITY  ARGUMENT 


If  8  of  (1.1)  is  finite,  in  addition  to  and  independent  of  all  the 
random  variables  defined  in  section  1,  let  {X*,  n  =  0,  1,  .  . . }  be  a 
Markov  chain  with  state  space  J,  transition  probability  for  i,  j  e  J  of 


p.  .  *  P(X  =  j  |  X  =  i)  =  "  .  ^7  n , 
*13  n  J  n-1  ’  3  i  31' 


and  initial  distribution  of  given  by 


P(XQ  =  i)  =8'  TTimi  • 


We  also  define  Z  (x)  bv 
n 

<w=f(n)(x;  Vi(Cd- ■•-,,o'V)=8(n)(x;  Cr  C 2 . V 

★ 

as  defined  in  (1.3).  The  relation  between  Z^(x)  and  Zn(x)  is  given  in 
the  theorem  below. 


THEOREM  2.1.  For  each  P>  0  and  each  r  >  0,  there  is  an  *'  where  for  all 


:■(  sup  I  Z  (0)  -  7  (X)  !>  c)  <  e  . 

|xl?r  ‘  n 

In  addition,  if  (x„,  Z  (0))  converges  in  distribution,  then  (X*,  Z 
n  n  -  n 

converges  in  distribution. 


PROOF.  Let  Xq  have  initial  distribution  -,  and  let  K  be  a  finite 


subset  of  J ,  where  both 


£  tt.  >  1  -  e  and 
ie  K 


tt  .  n . 
l  i 


>  1  -  e. 


From  (1.4),  for  B>0,  e>0,  we  can  select  an  N  where  for  n>N, 


P(  sup  |  2  CO)  -  Z  (x)!  >  e)  <  e  min  (tt  ) . 
IxUB  "  n  ieK  1 


Then  P(  sup  |  Z  (0)  -  Z  (x)  |  >  e) 
IxIsB  n  n 


l  P(  sup  IZ JO)  -  Z* (x)  I  >e|X*«  i)B_1ff  n 
ieJ  |x|£B  n  n  u 


=  l  P(  sup  I  Z  (0)  -  Z  ('x)|  >  el  X  =  i)B  ^.m. 
icJ  |  x  l<  P  n  n  °  11 


=  l  ttT1P(  sup  I  Z  (0)  -  Z_Cx)|  >e,  X=i)B_1TTm  .  (2.1) 

ie  J  |xl<"  n  1 


From  Kemeny,  Snell  and  Knapo  (1976),  we  have  for  ie  K, 

P (  sup  |  Z  (0)  -  Z  (x)|  >  e,  X  * i) 

|xl<B  n  n 


=  P  (  sup  |  Z  (0)  '  Z  (x)l  >  e  ,  X  =  i)  <  e  min(r . ) 
lx  Is  B  n  ieK 


Thus  from  (2.1)  and  the  definition  of  K  it  follows  that 


*  *  r  ~  1  '1 

P  (  sup  |Z  (O’)  -  Z  (x)  |  >  r  ]  <  T  tt.  e  min  (tt  . )  •  S  tt  .m  .  ♦  c  <  2e . 


Also,  again  letting  have  distribution  it  ,  we  have  from  theorem  1.1 

that  (Xn,  2,(0))  converging  in  distribution  implies  for  all  continuity 
points  of  y, 

^2P(*0  =  i’  V0)  £y)=pO^0  =  i>  2  £y),  for  some  Z. 

Therefore , 

tU®  P(X0-i'  2n(0)  -y)  =  li™  ^(2^(0)  <yj  =  i)g 

sjU2  p(2n(°)  syl  Vi)r  2<v)e'1m.. 

As  such,  we  have  shown  that  (X*,  Z*(0))  converges  in  distribution.  0 

Tne  fundamental  lemma  of  this  paper  makes  clear  the  relation 

★ 

between  {Xn}  and  the  behavior  of  the  continuous  time  semi-Markov  process. 


PROOF. 


For  a  particular  k,  let  us  define  a  new  Markov  rene /al  process 


((Y  ,  T  ) »  n  =  0 ,  1,  ...}  with  state  space 


Tk  +  1 


*={(V  *i . ik)cJ  Pi.i/Pi,i  •••  Pi  *  >0} 


1 01 1  Jl*2 


k  -  lk 


hy 


P(Yn  "  V  ••  •  '  V  ’  Tn  Tn  -  1  1  Yn  -  1  =  (10’  V  -  1k)) 


=  a  (t)  n  Trip* u  .) 
Vk  1=1 


As  is  apparent,  {(Y^,  T^) }  is  nerely  the  process  { (xn »  Tn)}  with  memory 
of  the  previous  k  states  it  has  visited.  Therefore,  for  N(t)  >k, 


P(XN(t)  -  k  =  V  XN(t)  -  k  ♦  1  5k  -  1 ’  ' • • *  XN(t)  =  V 

=  PfYN(t)  3  (lk’  lk-r  "  *  ’  l0”  ' 


Also,  (Yn)  must  be  aperiodic  and  irreducible,  since  {X^}  is.  Finally, 
letting 


ft,.  .  .  .  =  v .  p. 

(j^>  J j *  •  •  • »  Jp  -7rp l 


-  l 


we  have  that 


-10  - 


=  y  (it.  T).  .  r> . r  •  •  V'  . 

iQe.T  10  Vo  303  1  3k  -  23k  -  i  3k  -  l3k  ^O’ 


As  such,  we  have  from  Karlin  and  Taylor  (1975)  that  Y  is  positive  re- 

* 

current  with  stationary  measure  it  .  Therefore,  from  Cinlar  (1975). 

> 

it  follows  that 

V)  P(Xy(t)  -  1  =  V  Xv(t)  -  k  +  l  =  1k-  1  ’  *•**  X\(t)  =  V 


\  ^  n  • 

(ik'  h-r  V51'  lk  Vk- 

=  g~  3  T.  P.  .  ...  r’.  .  r  .  . 

lk  1k1k  -  1  Vo  x0 


S'-. 

ii1o  1o 


The  proof  is  completed  once  it  is  observed  that 


pCxo  =  io’ 


*  -1 

X,.  =  ij  =  6 


\  1k1k 


1i1o  xo 


Thus  we  see  that  in  continuous  tire,  the  Harkov  renewal  process  can  in 
a  sense  be  1  reversed"  to  look  like  the  dual  Harkov  chain  with  a  different 
initial  distribution. 

From  this  lemma,  we  can  establish  the  main  theorem  of  this  paper. 
THEOREM  2.3.  If  8  <  «,  if  (':  ,  Zn(2p) )  (X ,  Z) ,  and  if  for  all  e  >  0, 

there  is  a  B  where  for  all  k>  0  lim  P(!  Z  ...  _  ^1  >  E)  <  r  ,  then. 

t->co  k  ' 

lim  P(X  (t)  =  i ,  Z(t)  £y)  =  lim  P(X*  =  i,  Z*(0)  <y),  for  all  continuity  points  y. 

t-KO  ‘  11 


PROOF. 


Let  (Xq,  Zn(0))  have  limiting  distribution  (X^,  Z  ),  and  let  (i,  y)  be 


continuity  point.  For  any  e>  0,  select  a  p>  0  where  lim  P(|Z,.,  ,  ,  |>p)  <e 

-  .  -  '  v.  t )  -  K 


f-x» 


for  all  k.  We  know  from  theorem  2.1  there  is  an  N'  w’hr-re  for  nil  n>Y. 


P(  sun  I  Z* (x)  -  Z*(0)|  >£)<£. 
!x|<B  n  n 


(2.2) 


Select  a  k>  Y  where 


|P(X*  =  i,  Z*  (0)  <y  +  e)  -  P  (X*  =  i ,  1  <  v  +  E)|  <e. 


(2.3) 


From  the  i.i.d.  nature  of  {U’n(i)},  we  can  see  that  for  g^(:<;  i^,...,  in) 
as  defined  in  (1.3)  that  for  all  k 


Z\l(t)  g^(Z*:(t)-k;  X,’(t)  -  k  *  I’’  •  •  ’  X?’;t)} 


Tlierefore 


.00 


P(X?i(t)  =  1  *  ZN(t)  “  P(XY(t)  =  x’  F'  (Z,!(t)-k;  Z>;(t)  -k +  XN(t)5 


-f>(^v)rf.'\  =  i»  g  (^Mf*  -V  _  1/  >  !»•••»  ~y  *  I  _  1  I  “ 


:(t) 


*M(t)  -  k  ’  ’.’(t)  -  k  +  T 


•(t) 


YCt)  -  V 


00  ,v. 


SxWb'8  (X;  X 


!(t)  -  k  +  1 


.00 


’  •  •  •  ,  -  E  C°;  X:!ft)  -  k  +  1’  ■  ’  •  *  Xy(t))  1  "e} 


(t)  -  k 


(t) 


+  P  ^  ZN  (t  )  ~  k1 


+  P(  sup  |g(  ^  (x;  .  J.  +  J .  X,,(t))  "  )(0;  x”(t)  -  k  4  !»•••»  X’J(t)5  1  >  e) 


Tt  is  t’  crc^or^  cl^ar  t’nt 


(k) 

=  i  rr  v  J 


P^X N(t)  =  1  ’  ZN(t)  "y^  -P^X,:(t)  =  1’  g  (0;  XV(t)  -  k  *  1’  ‘ '  ■  ’  X”(t)->  -y  +  £) 


+  P(|ZN(t)-kl>r'^ 


+  p(  Slip  |g(  ^(x;  .  k  +  !»•*• »  XN(t))_g(  ^0;  xr!(t)  -  k  +  1 .  XN(t))l> 

l  x  ( is  * 


We  first  note  that  lim  P ( 1  2  ^  .  kl>P*)  <  e  • 

t-*»  '■  ' 


Also,  since  the  possible  values  of  X,,.  .  .  X. are  countable,  and 

N(t)  -  k  +  1  N(t) 


for  all  i  ,  i2 .  i  e  J  , 


P(X^(t)  =  1’  ^°:  XN(f)  -  k  +  !’•*•  ’  XM(t)^“y  +  e 


X*’(t)  -  k  +  1  =11 


•  •  X, 


Kt) 


is  a  constant  bounded  by  1,  we  have 


P^XN(t)  =  i  ’  (°:  XN(t)  -  k  +  r  ' '  ■  ’  X‘!{t)')  Sy  + 


=  P(X*  =  i,  g(k)(n;  X*_  2 .  X*)  £ye). 


From  a  similar  argument  we  have  from  (2.2)  that 


lim  P(  sup  lg(  }(x;  XN(t-}  _k+  xvrt))  *  P(  }(n'  Xv(t)  _k  +  !»•••>  x,  (t))  (>  ^ 


:(t) 


’(t) 


(t) 


(t) 
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Thus,  combining  (2.3)  and  (2.4)  we  have  that 
Tim  p(xN(t)  =  1  -  zv(t)  sy)  -p(xn  =  i>  1  +  e)  +  2c. 

By  similar  arguments  it  can  be  established  tha* 

P(X*  =  i,  Z*<y-c)-2e  <  lin  P(\. ,  =  i ,  Z..  £y)  . 

t-w»  ^  ' 

which  completes  the  proof.  C 

Of  course,  the  rrucv'l  condition  in  the  application  of  theorem  2.3 
to  establish  that  if  C  < 00  then  (X^,  Z^)  converging  implies  (”(t)  ,  7(t")) 
converges  i  the  condition  that  for  all  k 

Tim  P(  lZ..m  _  y  I  >  r ■)  <  c, 

t-Ko 

which  implies  that  a  bound  can  be  selected  that  will  apply  for  any  k. 

We  now  establish  that  for  semi-Markov  ■•rr'ce^r.  '■  c1  •11"’  't^-'H*  st  '*<> 
space  with  a  regeneration  point  with  finite  expected  return  time,  that 
such  uniform  ’  "i.m  !s  *»yist. 

let  (Y  ,  T  }  be  a  well-defined  Markov  renewal  process  on  some  normed 
n  n 

state  space  S  (for  a  more  detailed  definition,  see  Cinlar  (1969)). 

> 

usual,  we  define  Y(t)~Y  .  ,  and  for  any  ve  S ,  random  variable 

vT ) 

Z  and  event  A,  we  define 


Hy(2)  »E(Z|Y0  =  y),  and  Py(A)  =  P(A|Y0  =  y) . 


We  then  assume  that  there  exists  a  point  xn  c  S  where  for  all  yeS 


P  (  u.  (Y  =  xn))  >  0, 
y  n=l^  n  0'J 


(2.5) 


and  for  V  -  inf  {n:  Y  =  x„)  and  T  =  T 

n  0 


E  GO 

xo 


(2.6) 


E  (T)  <■». 

0 


(2.7) 


As  is  shown  in  Cinlar  (1975),  such  regeneration  points  make  the 
behavior  of  semi-Markov  processes  quite  tractable.  In  particular,  we 
have  the  following  theorem. 


THEOREM  2.4.  Let  (Y  ,  T  )  be  a  Markov  renewal  process  on  a  norned  sp 


S  where  there  exists  an  xQ  c  S  satisfying  (2.5)  ,  (2.6)  and  (2.7) .  Ther 
for  all  e>  0,  there  is  a  r>  0  where  for  all  k>  0 


Tim  P(|  |YV  _  k  |  l>b)  <  c. 

t-K»  '  V  1 


PROOl  . 


As  is  shown  in  Grey  (1971 )  -  we  have  that  P  (Y^«xn  i  .  o. )  =  1 


for  almost  all  y,  and  there  exists  a  stationary  probability  measure 
u(-)  where  for  all  AS, 


w(A)  =/su(c!x)P(Yl£  A|Yn=x). 


(2.8) 


Therefore,  for  almost  all  yeS, 


S:  VV) 


!  I  >  B)  =  lim  P  (!  I  Y„ 


t-K» 


(t)  -  k' 


*), 


and  since  lir  N(t)  a.s.,  ve  need  only  consider 


Ji:y"  No 

Clearly  x^  is  a  regeneration  point,  and  therefore 
pXqCI  |Yv(t)  _  kM  >  n,  m (t)  a)  =  Px0C1 1  Yr  (t)  -k1 1  >  r>  ‘  w  2k*  T>  t} 
+  PxQ(l  |Y"(t)  -kM>  '  '  *(t)  -  k*  0  S’’U)  -r(T)  <M 

+  ^0Px0CI  !YM(t)  -kH  >  B’  ’4t)  -  ::(s)  >k|T=  s)dPx  (T  <s)  . 

Since  T  is  a  time  of  regeneration,  we  have, 

/oPx  (I1  Vrti  -  k!  1  >  B*  ‘ 'CO  5  k>  -Ct)  -  V(s)  >k)  |T  =  s)dP  (T  ;s) 

A0  N  Xq 

-  I>*0"  1  YM(,  -  s,  -  k!  !  *  r-  '  a  -  s>  2  »)«•  (T  is)  • 

Therefore  we  have  that  (2.9)  is  a  renewal  equation,  and  if  it 


be  shown  that 


(2.10) 


Px0(|r\'(t)  -  fc1  (  >  N’(t)  -  k’  T>  t) 

+  Px  (l  lVt)  -kf!  >  B>  N(t'  “k>  0  "N(T)  <k> 


is  directly  Rienann  inferrable,  then  the  basic  renewal  theorem  will 

yield  lin  P  (i  |Y  .  kU  >  Y.,  N  (t)  >k). 
t-+°°  *0  *  ^  * 

Clear' y,  the  first  tern  of  (2.10)  is  directly  Rienann  integral) le,  since 

V!V)-k">3-  T‘ «>  <r,' *» 


which  is  nonmcreasi np  with  /” P  (T>  t)dt-E  T  <■ 

x0  x0 


Also  P  (I  I  YN(t)  ,ki  I  >  3,  K(t)>k,  0  <  Ut)  -  •■IfT)  -  k) 


<P  (T>  t)  *P  C-'(t)  -  N(T)  <k.  T  St) 
X0  X0 


=  1  -  P  (N(t)  -  ::(T)  a,  T  St). 
X0 


Because  P  (Nft)  -'!(T)^k,  T  st)  is  nond  croasi;:  • .  if  we  can  show 
X0 


fn'.P  (T>  t)  P  C:(t)  -\(T)  <k,  T  <t)jdt  <»,  we  will  have  establislied 

x0  xo 


the  directly  Rienann  iriicerabi  1  i  ty  of  the  second  term  of  (2.10).  First 
note  that  fZ P  (T>  t)dt<°°. 

u  Xrt 
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Also 


Jn Pv  -N(T)  <k,  T  <t)dt 

u  xQ 

=  /n/opY  <”Ct  -  s)  <k)dP  (T  <s)dt 
u  u  x()  x0 


L(LPV  (N'ft  -  s)  <k)dt)dP  (T  <  s) 
"o  xn 


-/oC/>v  (Tv  >t)dtV.V,  (TSr;)  -7  T, 


Since  E  T<®,  it  clearly  follows  that  E  T,  <  kE  T  <co,  which  establishes 
x0  <1  K  x0 
the  direct  Riemann  inteprabi lity . 

Therefore,  from  the  basic  renewal  theorem,  we  have  that 


lim  P 

t-w  x0 


(I  i  Y, 


N(t)  -  k 


I  I  >  B) 


=  (E  T)_1L/ 


orxQ(  1 1  Yr:  ( tY 


-  k 


.11  >  B  ,  C t )  >  k  ,  T>  t)dt 


/'opx  <' iY, 


N(t)  -  k 


I  I*  R,  N(t)  >k,  OS'N(t)  -  NTH  <k)dt  J.  (2.11) 


7 '  establish  the  uniformity  of  the  bound  B,  let  us  define 


v  =  sup  (inr  !i). 
ft  t<T  v 


f 


Since  E  T  <  ^  and  Y  <«  3.s.,  we  have  from  the  dominated  convergence 

v  ’ 


theorem  that  lim  J” p  (Y  >B, 


T>  t)dt  =  0. 


Therefore,  for  all  e>  0,  there  is  a  D >  0  where  for  all  k>  0, 


fto(l|Y,t)  .  kM>  B.  vO)*k,  T>  t)dt  </“Px  (Y*  >  B,  T>  t)dt<£ 
Also, 

#x  (IIX,(t)  .  kll>  B-  ^O)  2k«  0  s*(t)  -W(T)  <k)dt 


</“p  (Y*  >  B,  r:  (t)  >k,  ‘  (t)  -v  (T)  <k,  t  <t)dt 

x0 


=  In  In  O  >  B,  V  (t)  >k,  M(t)  -•  (s)  <  k  t  T  =  s)  dP  (T  <s)dt. 
U  0  x9 


From (2. 13)  it  follows  that 


^Px0(1|Y 


N(t)  -k 


\  >n,  N ( t )  >  k  ,  0  r.  Mft)  -  N(T)  <k)dt 


k-  1 


<-r0  In  l  Px  O'  >B  >  •’  (s)  >  k  -  j  , 

j  =0  xo 


•:(t)  -  "(s)  =  j  I t  =  *■  )dp  p  <  ?)  it 

xn 


k- 1 

In  J"  I  Px  O  >  B,‘  (s)  >  k  -  j  |T  =  s)P  r(t-s)*j)dt  (*\  (T  < 
j=o  x0  xo  -0 


k-1 


C  l  P  O  >  B,  ‘(s)  >k  -  j|T  =  s)L/”p  r(t)  =  j)dtjdP  -(T  <s) 
j=0  X0  X0  0 


It  is  easily  seen  that 


O'  C(t)  =  j)dt  .Or 


(T.  . >  t)  -  P 

.  J  ♦  1  xr 


(T.  >  t) jdt 


V 

.J* 

0 


+ 1 


V‘>*> 


where  Pn(xQ,  A)  =  PfY^ e  A fYQ  =  x) . 


-  i ,  oyjt  1 . , 
j  '  S  0  '  y  1 

From  (2.8)  it  follows  that 


/sy(dx)/cPn(x,  dv)EyT1  =  frv  (dx)r:yTj  =  y  (>0)FX  T  (the  last  equality  can 
be  found  in  Cinlar  (1975)). 

Therefore,  u(xQ)Ex  (T.  +  ^  )  <p(x^)Ex  T,  and  we  once  again  find  that 


(T.  _  -  T.)  <E  T 

n  J  *  1  J  x 


Therefore,  from  (2.14)  we  have  that 

/oPxQ(l  |YN(t)  _  k  I  I  >  B,  N(t)  >k,  0  <V(t)  -"(T)  <k)dt 

k 

£(E  T)  I  />  (Y  >  B,  x  (s)  >  j  |T=  s)dP  (T  is) 
o  j=i  o  o 


=  (Ex  T)  l  P  (Y*>  B,  \'(T)  >  j) 
0  j  =  l  0 


S(E  x  T)  l  Px  (FT(T)  >  j)  =  (E  T)E  v<».  (2.15) 

0  j=l  *0  *0  x0 


Again  by  the  dominated  convergence  theorem,  for  all  e>  0,  there  is  a 
8>  0  where 


(C  T) 


l  P 


(Y 


>  2,  N(T)  >  j)  <  e. 


From  (2.15)  it  then  follows  that  for  all  k>  0, 


/qPx  ('!  _  k  I  I  >  B,  r(t)  >k.  0  <\f(t)  -  V(T)  <k)dt  <£, 

and  therefore  (2.11)  coupled  with  (2.12)  and  (2.15)  completes  the  proof.  C 

Of  course,  it  must  be  pointed  out  that  should  {(X^,  Z^)}  have  a 

point  of  regeneration  the  convergence  of  (X(t),  2 (t ) }  follows  directly 

from  the  basic  renewal  theorem,  and  theorem  2.3  is  unnecessary.  This 

situation  is  not  the  primary  situation  of  interest.  But  even  in  this 

case,  as  will  be  shown,  the  theorem  can  lead  to  a  more  satisfying 

answer  to  the  limit  behavior  of  (X(t),  Z(t)}  than  the  generally  intrac- 

tnl  le  integral  answer  which  results  from  the  basic  renewal  theorem. 

The  primary  situation  of  interest  is  when  (X  ,  Z  }  has  no  point  of 

n  n 

regeneration,  but  has  instead  the  following  two  properties 

1)  Zn(x)  >0  a.  s  .  ,  Vn ,  Vx , 

(2.16) 

2)  if  x>y,  then  Zn(x)>Zn(y)  a.s.. 

In  this  case,  we  will  show  that  should  (X^,  Z^}  converge  in  distribution, 

the  uniform  boundedness  condition  is  satisfied  Pv.  bourtdih'-  {X  ,  Z  }  by 

n  n 

another  chain  (X  ,  V  }  with  a  regeneration  point. 

n  n  "  1 

THEOREM  2.5.  If  B  <  ”,  {Xfi,  Z}  ,  and  if  property  (2.16)  is 

valid,  then 

lir.  r ( X ( t )  =  i,  Z(t)  < y)  =  lim  (X*  =  i,  Z*(0)  <y), 
t-*»  n-**’ 

for  all  con t inui ty  poi nts  y . 


•  A  k 
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PROOF .  If  there  is  no  regeneration  point,  then  from  theorem  1 . 1  we  know 
that  for  some  i^,  and  any  x>0,  there  is  an  A>0,  a  P>A,  an  e>  0,  and  an 


m  where 


sup{D :  linP(X  =  i _ ,  Z  (x)  <  0)  =  0} 
n-^00  nun 


(2.17) 


P<W  *».»“«•  ••••  W  W’" 


Assume  without  loss  of  generality  that  A  =0.  We  can  then  define  two 

new  Markov  chains  {(X  ,  )}  and  (X  ,  V  },  where  the  transition  probabilities 

v  n  n  n  n 

for  (X^,  W  }  aic  given  for  arbitrary  set  C  by 


P(X  =i,  W  cr  IX 


n  -  1  J  '  n  -  1 


P(Xn  =  i-  Zncf  1  Xn  1^’  Zn  -  1  =  y>  if  *  * orC"r°’  B '!  = 


P(Xn«i0,  Zn  e  C  u  r  0 ,  B]  I  Xn  ,  =  j.  Zn.  J*y)  if  i  =  V  BeC  (2'1S'1 


if  i  =  i0,  0  c  rn,  p)  , 


and  those  for  {X  ,  V  }  are  ?iven  bv 
n  n 


^ <V*’  VCIX„-l'>'  Vl*v’ 


>  (*„-!•  2n  «  C  1  -  1  -  •»  *  h.l’rt  K  “*0  or  rn!°'  *>» 

pfx  =  i_,  z  c  r  u  rn,  b:  IX  =  i .  z  .  =  v)  if  i  =  iA,  o  e  C  (2.19) 
nO  n  n  -  1  n-1 


From  the  above  definitions,  we  can  see  that  '''  and  V  are  merely 

n  n  } 

versions  of  2  in  which  whenever  X  =  i_  and  Z  sB,  the  value  of  Z  is 
r.  n  0  n  n 

immediately  changed  to  B  and  0  respectively,  and  then  the  process  is 
restarted. 

Clearly,  from  (2.16)  it  follows  that 


V  (x)  <Z  (x)  <W  (x)  a.s.. 
n  n y  J  n 


Let  us  define 


"z  ~  inf  {n>  1:  V  V  Zn  *B}> 


=  inf  {n>  1:  X  =in,  <  P. }, 
w  n  O  n 


{:1>  I:  vv  V  **>• 


From  Cinlar  (1975),  we  need  only  show  that  I  •  <»  to  have 


(X^,  1“  )  satisfying  theorem  2.4.  The  other  condition  follows,  since 


F „.N  < '-implies  tiierc  is  a  stationary  probjhi  litv  measure  ..  with 
(10,  b)  W  '  • 


y(ig,  B)  >  0,  in  which  case  for  T  =T  , 


,*1. 


n(i(J,  B)VU(i0’  Jy^F(i,  y)VU(V  B)]  X2< 


Should  {X  ,  Z  }  be  er,  odic  tills  would  follow  immediately,  since 
n  n  J 


,■  =  F  . .  \  .  Unfortunately  without  a  f-i  rreducihi  litv 

1 0 ’  ^  A  (V  ^  z 


condition  (for  definition,  sec  Grey  (1 971 )  J  ,  the  convergence  of  (X  , 


is  not  sufficient  for  ergcdicitv.  however,  it  is  clear  from  (2.16) 
that  for  any  x  < y ,  and  for  x>  B  that 


Thus,  it  is  sufficient  to  show  that  for  some  x>  B,  E,.  ,N  <"• 

Let  us  first  establish  that  (X  ,  V  }  is  ergodic.  Since  (in, 

n  n  0 

is  a  regeneration  point,  it  follows  from  Tweedie  (1975)  that  if 

lim  P(Xn  =  iQ,  Vn  =  0|X0  =  i,  V0  =  z)  >  0  for  some  (i,  z)  ,  then  (X^, 
n+<” 

is  ergodic.  From  (2.16)  and  (2.19)  it  is  clear  that 


P<VV  V0IX0  =  i-  Vz>iP(VV  ZnSBIX0=i,  Z0  '  z>  ‘ 


Therefore,  we  have  (X^,  V^}  is  ergodic  since  for  all  (i,  z) , 


liJD  P(xn»  V  "n  ~BlXn  =  i.  Zn  =  z)  >  e. 


n-»“ 


From  the  ergodicity  of  (X  ,  V  },  it  follows  that  E  N  <  « 

n  n  (i ,y  0)  v 


It  can  be  shown  that 


m 


:(v  °)Nv=  JilP(V  °)(Nv= 


P(in.  0)<Vr\L#?L  0) (j '  dx|Nv>r.)(r  +  F.u>  ^ 


jeJ  v'  ^G’ 


From  (2.17)  it  follows  thatE^  O)!'v<0°  il?,Plies  that  E(j  xy\<' 


(m) 

almost  all  x  in  the  non-empty  support  of  P> 1  Q ( i Q ,  (B,oo)l’.'v>  m) , 


RV’w 


Wliich  in  turn  implies  that  E.. 


< “  from  (2 . 20) . 


-2A 


Since  E,.  _.N  <a,t  we  have  that  theorem  2.4  applies  to  {X  ,  W  } 

1^0 9  " J  w  n  n 

Therefore,  for  any  e>  0  there  is  a  B>  0  where  for  all  k>  0 


lim  P(|W?.  _  k  |>  B)  <  e. 

t-Ko  ‘  '  ' 


Since  Z^Cx)  <Kn(x)  a.s.,  this  in  turn  implies 


lim  P(IZ,  _  kl  >  B)  <  lim  P(|V,'N  _  k  I  >  B)  <  e, 

t-y°°  ^  t-*°°  1  ^  ) 


which  from  theorem  2.3  ccrpleUs  the  proof.  □ 

In  the  next  section,  we  will  apply  these  results  to  several 
storage  models  to  determine  the  convergence  behavior  in  continuous  time. 


3.  APPLICATIONS 

To  illustrate  the  applications  of  the  theorems  in  the  previous 
section  we  will  examine  the  behavior  of  several  storage  models.  The 
following  notation  will  be  necessary.  For  any  random  variable  Y  define 
Eff(Y)  by 

Er(Y)=  5>  E(Y|Xn  =  j) 
jeJ 

where  *  is  the  stationary  measure  of  the  chain  { } . 

EXAMPLE  1. 

Let  (Un(i),  n  =  0,  1,  ..,}  be  an  i.i.d.  sequence  of  real-valued 
random  variables,  independent  of  {X^}  and  of  (Un(j)},  for  j  x  i . 


Let  E  |  U,  I  <»,  and  let  E  !J  <0.  iv’e  then  define  our  contents  in  storage 

7T  1  r  i 

Zn(x)  recursively  by 

Zn(x)  =max(0,  Zn  _  ^x)  ♦  L'n(Xn)), 
and  ZQ(x)  H  x* 

Of  course, 

Z(t)"ZN(t)(x>- 

This  model  is  based  on  an  early  model  for  dam  theory  proposed  by 
Moran  (1954),  which  has  shown  itself  to  have  diverse  application  (for 
example,  in  waiting  times  for  queueing  theory).  In  the  present  form, 
it  has  been  examined  by  balagopal  (1979),  Puri  (1978),  Senturia  and 
Puri  (1973,  1974),  Puri  and  Woolford  (1981),  and  Puri  and  Tollar  (1985). 
In  its  most  general  form,  Puri  and  Woolford  (1981)  had  shown  it  converged 
in  distribution  when  appropriately  normalized  when  E^.Uj^O,  and  hypoth¬ 
esized  it  should  converge  without  normalization  when  E^l'^  <0.  This 
was  shown  to  be  true  by  Puri  and  Tollar  (1985),  who  illustrated  that 
{X  ,  Z  }  must  have  a  renewal  point  (i„>  0),  in  which  case  renewal  theory 
directly  yields  for  T  =  inf  (t>  Tj :  X(t)  = iQ,  Z(t)  =  0}, 


lira  P(X(t)  =  i,  Z(t)  e  A)  = 

t-*oo 


[En  m(x(t)-i.  Z(t)eA,  T>  t)dPr.  m(T<t). 


Noting  that 


n  n 

Z  (x)=max  (  max  (  £  U. (X. )) f  x ♦  £  U. (X. )) , 

l<j<n  i=j  +  l  11  i=l  1  1 


(3.2) 


we  see  that 

j 

Z  (0)=  max  (  l  U.(X.)). 

-1  £j  <n- 1  i=0 

n 

Since  E.  <  0 ,  we  have  from  Chung  (1967)  that  £  U.(X.  )-*•-»,  a.s.,  in 
11  i=0  1 

which  case 

J 

(V  Zn}  — ' '(V  SUP(  i  MM)  a.s.. 

-l<j  i=0  1  1 


From  theorem  1.1  this  implies  (X^,  Z^)  converges  in  distribution  if 
we  also  observe  from  (3.2)  that 


n 

sup  |Zn(x)  -  Zn(0)|  <max(0,  B  ♦  ^  ^(X^  +  O  a.s.. 
lx|£B  i=l 


Note  that  Zn(x)>0  a.s.,  and  if  x^y>  then  Zn(x)>Zn(y)  a.s..  As 
such,  we  have  the  following  theorem  as  a  consequence  of  theorem  2.5. 


THEOREM  3.1.  If  B<»,  E  U.  <0,  then 

-  Tf  1  - 

*  i  * 

lim  P(X(t)  =  i,  Z (t)  <y)  =  P (X  = i ,  sup(  £  U.(X.))  <y) 
t-*-  u  -l<j  i=0 


for  all  continuity  points  of  y. 


It  should  be  observed  that  this  form  is  substantially  more  satis¬ 
fying  than  the  integral  form  of  (3.1)  obtained  by  Puri  and  Tollar  (1985) 


While  this  by  no  means  allows  us  to  actually  compute  the  distribution 
of  the  limit,  Puri  (1978)  has  found  the  Laplace  transform  of 

sup(  f,  U.  (X.))  for  a  two-state  Markov  chain  (X  },  which  is  a  least  a 
Q<  j  i=l  1  1  n 

step  in  characterizing  the  limit  distribution. 

EXAMPLE  II. 

Let  ((L'n(i),  V  (i)),n  =  0,  1,  . .  . }  be  an  i.i.d.  sequence  of  bivariate 
real-valued  random  variables,  independent  of  -( X^ }  and  of  Vn(j)} 

for  j  *i. 

We  then  define  a  proportional  allocation  scheme  for  our  contents 
in  storage,  zn(x),  recursively  by 

(5-5) 

and  ZQ(x)  =  x, 
and  again  Z(t)  =  Z^.  ^  (x)  . 

While  this  model  has  received  no  attention  in  the  general  frame¬ 
work,  it  has  proven  to  be  of  interest  in  the  simpler  case  where  we 
define 

ZnM=VZn-l«*'V 

for  {(U^,  W^) }  an  i.i.d.  sequence  (see,  for  example,  Barnard  ,  Schenton, 
and  Uppuluri  (1967),  Paulson  and  Uppuluri  (1972),  Vervaat  (1979)). 


Of  particular  interest  is  the  observation  that  the  techniques  for 
semi-Markov  processes  on  arbitrary  state  spaces  of  Cinlar  (1969),  Athreya, 

f 

McDonald  and  Ney  (1978a, b)  and  others  have  little  chance  for  success  on 


this  model,  since  there  is  no  guarantee  that  the  Markov  chain  (X^,  Zn(x)} 
is  <{>-irreducible.  As  such,  while  one  can  show  the  Markov  chain  converges 
by  a  reversal  argument,  and  therefore  apply  theorem  2.5,  ergodicity  of 
(Xn,  Zn(x)>  cannot  be  demonstrated,  so  the  typical  starting  assumption  of 
semi -Markov  processes  is  missing. 

We  first  establish  a  theorem  on  the  convergence  of  (X^,  Zn(x)}. 

THEOREM  3.2.  If  E  £n|U.|  <0,  E  (£n|V,|)+  <-,  then 

li^p(xn=i,  Zn(x)  < y)  =  P(XQ  =  i,  Z  £  y) ,  for  all  continuity  points  y  where 

oo  i- 1 

Z«  l  V  (X  )  n  U  (X  )  <»,  a.s . . 
i=0  1  j=0  1  1 

PROOF.  From  (3.3)  it  is  easily  seen  that 


n  n  n 

z  (x)  =  x  n  u. (x.)  +  7  v.  (x.)  n  u.(x.), 

n  ..li  .L,  ii.  •  ,  j  J 

i=l  i=l  j=i+l  J  J 

so 

a  n-1  n-1  i-1 

z  (x)  =x  n  u  (x  )  ♦  l  v  (x  )  n  u.(x.). 
i=0  1  1  i=0  1  1  j=0  3  3 


k  k 

Since  Zn\  H  U.(X.)|=  ^  £n|U.(X.)|,  we  have  from  Chung  (1967)  that 
i=0  1  1  i=0  1  1 


E^fnlUjI  <0,  implies 


lim  -  ln\  n  U.  (X. )  I  <0,  a.s.. 
n-  n  i=0  1  1 


(3.4) 


Therefore, 


sup  |Z  (x)  -  2  (0)|  -B  n  |U.  (X  )  |  +0  a.s., 
|x|<P  i«=l  1  1 


so  condition  (1.4)  is  satisfied.  Therefore,  from  theorem  1.1  we  need 
only  examine  Zn(0). 

From  (3.4)  we  have  for  all  e  >  0,  there  is  a  6,  0  <  6  <  1 ,  and  an 
N  where  for  n  >  N , 


P (£n I  n  U  (X.)  |>  nfnfi ,  V  n  > N)  - 
i=0 


P(l  n  U. (X  ) | >  6n,  V  n  >  N)  <  e . 
i=0 


(3.5) 


Therefore , 


P( lzm(°)  ~  2n (0) |  >  e ,  v  m  >  n  >  N) 


*P(  l  IVjCx.)!*!  n  u  (X  )|>  e) 

i=n  j  =0  J  } 


<P{  l  IV  (X  )|6n  +  1>  e)  +  e, 
i=0 


(3.6) 


where  the  last  inequality  follows  from  (".3)  and  from  {Xn>  having  the 
stationary  distribution. 


A 


For  any  x>  1, 


oo  OO  CO 

l  P( IV  (X  ) | >  x1)  =  l  V  l  P(|V  (j)|>  X1) 
i=0  1  jeJ  Ji=0  1 


=  I  I  P(-enlV.U)  l>  i Znx) 
jeJ  Ji=0 


<  l  „ AZnxyl(l  +  EUnW ,(j)l)+) 
UJ  J 


=  (Cnx)  ^(l  +  F^CCn  |Vj|)  +  )  <•. 


As  such,  by  the  Borel-Cantelli  lemma,  for  any  x>  1 


P(|Vi(Xi) |>  x1  i.o.)  =  0. 


Therefore,  for  any  x  <  <5  1,  since  [  x161<“>, 

i=0 


CO  .  oo 

I  l  |V.(X.)  I61  -  l  X1^1  |  <c=,  a.s .  , 
i-0  i=0 


which  implies  that  £  |V. (X. ) | 61  < ®,  a.s.. 

i=0  1  1 


As  such,  we  find  from  (3.6)  that  for  N  sufficiently  large, 
P(|Zm(0)  -  Zn(0)  I  >  c ,  Vm  >  n  >  N  )  <e. 


rv.c  we  have  that  Zn(0)  converges  almost  surely  to  some  random 
variable  Z,  which  completes  the  proof.  [7 


To  complete  the  analysis  of  Z(t),  we  once  again  appeal  to  the 


results  of  the  previous  section. 

THEOREM  3.3.  I_f  8  <  ®,  |  U  1  <  0 ,  E^  (Ik  |V1  j)*  <*>,  then 

*  *  / 

lim  P(X(t)  =  i,  Z(t)  <y)=P(X„=i,Z  s:  y) ,  for  all  continuity  points  > 


★ 

1 


*  i-1  * 

y  V.  (X.)  n  U.  (X.)  <°°,  a.s .. 
i=0  1  1  j-o  1  1 


PROOF.  While  theorem  2.5  cannot  be  appealed  to  directly,  clearly 

n  n  n 

lzn(x)  I  -V:  x I)  =  lx'  n  IU.(X.) \*  l  |V.(X  )!  n  |u.(x)|, 

i=l  i=l  j=i  +  l  J  J 

where  {Xn>  W  (|x|)}  is  also  i,  Markov  chain.  Since  E^nlUjI  <0, 

E.^(-Cn|V^|)+ <«,  theorem  3.2  yields  that  (X^,  ( | x | )  - ■*-  (X,  W)  in 

distribution. 

Also,  property  (2.16)  is  clearly  satisfied  for  W  (|x|).  Therefore, 
proof  of  theorem  2.5  is  sufficient  to  establish  that  for  any  e>  0, 
all  k>  0,  there  exists  a  B  where 


lim  P C I Zr . 


t-X» 


:(t)  -  k1 


B)  <lim  P(|W  _  kl>  B)  <  e, 

t-KO  *•  ' 


•,  where 


the 

and 


and  therefore  theorem  2.3  can  be  applied  to  complete  the  proof.  0 


4.  CONCLUSION 


In  most  cases  when  a  storage  model  is  defined  on  a  Markov  renewal 
process,  the  convergence  of  even  the  discrete  time  version  cannot  be 
determined  from  general  Markov  chain  arguments.  As  such,  some  tech¬ 
nique  like  reversing  the  process  must  be  used.  Therefore,  if  tiie  con¬ 
ditions  in  section  2  are  valid,  we  essentially  get  the  convergence  of 
the  continuous  time  process  for  free.  Of  course,  it  is  clear  that 
conditions  such  as  (1.4)  and  (2.16)  car.  be  extended  directly  to  multiple 
dimensions,  and  the  results  will  remain  valid.  However,  for  more  general 
spaces,  the  counterpart  to  (2.16)  is  not  readily  apparent  to  us  at  present 
Crucial  to  the  usefulness  of  the  results  are  conditions  to  guarantee 
the  uniform  boundedness  of  Z^tj  j.  •  The  technique  of  bounding  (Xn,  Z^fx) 
by  another  Markov  chain  with  a  regeneration  point  seems  very  powerful . 

The  conditions  given  can  clearly  by  generalized.  For  example,  if  it  can 
be  shown  for  a  measure  -  that  t r  is  invariant  with  respect  to  (X^,  Zn(x)}, 
then  we  need  not  have  Zn(x)  S 0  in  (2.16)  for  the  results  to  still  be 

true.  It  should  be  noted  that  it  seems  that  Z  (x)  >0  should  be  untie e- 

n 

essary  in  (2.16)  ever,  under  the  presented  conditions.  However,  we 
have  been  unable  to  show  that  this  is  the  case. 

Certainly  there  must  be  methods  other  than  tnc  bounding  arguments 


used  which  could  be  considered  in  establishing  the  uniform  boundedness 

of  ,  .  Such  conditions  would  be  an  area  of  major  interest  in 

N  1 1. )  -  K 

the  applicabi lity  of  the  present  work.  And  of  course,  perhaps  uniform 
boundedness  is  not  the  only  condition  which  will  let  a  theorem  like 


theorem  2.3  be  valid. 
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